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@) Apparatus and method for selectively viewing video information. 
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(g) A television presentation and editing system 
uses dosed captioning text to locate items of 
interest. A closed captioning decoder extracts a 
closed captioning digital text stream from a 
television signal. A viewer specifies one or more 
keywords to be used as search parameters. A 
digital processor executing a control program 
scans the closed captioning digital text stream 
for words or phrases matching the search par- 
ameters. The corresponding segment of the 
television broadcast may then be displayed, 
edited or saved. In one mode of operation, the 
television presentation system may be used to 
scan one or more television channels unatten- 
ded, and save items which may be of interest to 
the viewer. In another mode of operation, the 
system may be used to assist editing previously 
stored video by quickly locating segments of 
interest. 
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FIELD OF THE INVENTION 

The present invention relates to television and 
other video information, and in particular to a novel 
method and apparatus for selectively viewing televi- 
sion or other video signals. 

BACKGROUND OF THE INVENTION 

Television has become the world's predominant 
communications medium. The variety of program- 
ming available on television has mushroomed in re- 
cent years with the growth of cable networks. In many 
areas, a viewer can select programming from among 
dozens of channels, broadcasting news, talk shows, 
movies, soap operas, serials, etc., from Geraldo Riv- 
era to William F. Buckley, from the banal to the sub- 
lime (or the reverse, depending on one's point of 
view). 

Although a large amount of information is broad- 
cast via television, the capability to selectively view 
television is primitive. Published television listings 
typically contain a minimal amount of information 
about the program to be shown, are often difficult to 
understand, and are subject to frequent errors and 
changes. Furthermore, a certain amount of television 
programming, such as a news broadcast, by its na- 
ture can not be described in advance. 

Most viewers would like greater control over what 
is being viewed. They may, for example, wish to view 
specific portions of a news program or a talk show 
dealing with a topic of interest They may wish to mon- 
itor multiple channels simultaneously. They may wish 
to screen out programming they regard as offensive. 

In theory, a viewer could record one or more tel- 
evision signals on one or more recording devices, 
such as commonly available video cassette recorders 
(VCRs), and review all recorded signals, editing such 
portions as desired. In reality, this would be an ex- 
tremely tedious and time consuming task. 

There has been some recognition of a need to 
provide a viewer with greater control over what is be- 
ing watched. For example, it is known in the art to 
place a small window in one corner of the television 
display, in which is shown the video signal of another 
broadcast channel, enabling the viewer to screen one 
channel while simultaneously watching another. As 
helpful as this feature is to the viewer, it is obviously 
limited in its capabilities. Screening multiple channels 
would require a separate window for each one. The 
screening must be done at the time the signal is 
broadcast. If something of interest is detected, the 
channel must be switched, possibly foregoing pro- 
gramming of interest on the channel first being 
viewed. 

Television viewers have accepted this lack of 
ability to screen, preview or extract portions of televi- 
sion programming only because no alternative has 



been presented. However, a need exists for ad- 
vanced capability on the part of the viewer to selec- 
tively control what is viewed. 

5 SUMMARY OF THE INVENTION 

It is therefore an object of the present invention 
to provide an enhanced method and apparatus for 
viewing video information. 
10 Another object of this invention is to increase the 

ability of a user of video information to control the 
content of the video presentation. 

Another object of this invention is to provide an 
enhanced method and apparatus for using televised 
is information. 

Another object of this invention is to increase the 
ability of a user of video information to find and view 
desired information. 

Another object of this invention is to increase the 
20 ability of a viewer of televised information to locate in- 
formation of interest from among a plurality of chan- 
nels and/or programs. 

Another object of this invention is to increase the 
ability of a user of video information to screen outma- 
25 terial which the user considers offensive or undesir- 
able. 

Another object of this invention is to provide and 
enhanced method and apparatus for using informa- 
tion in a vertical blanking interval of a video signal to 

30 find video matter of interest. 

In accordance with the preferred embodiment of 
the present invention, the information carried in the 
closed captioning portion of a television signal is ex- 
tracted and searched to find programming of interest 

35 to a viewer. The viewer can then view an entire pro- 
gram segment or edit segments. 

A conventional television signal comprises video 
and audio components. The video component of the 
signal is capable of carrying additional information 

40 during a Vertical blanking interval". In particular, the 
vertical blanking interval is used to carry closed cap- 
tioning information, which is a text stream. Typically, 
this closed captioning information is a simplified ver- 
sion of the spoken words being transmitted by the au- 

45 dio portion of the signal. While not a perfect represen- 
tation of what is being depicted by the television pre- 
sentation, the closed captioning text will generally 
contain words descriptive of a plot, news, or other in- 
formation being carried by the television signal. 

so A video presentation system in accordance with 

the preferred embodiment of the present invention 
comprises a suitably programmed general purpose 
digital computer, a television signal receiver/tuner, a 
closed captioning text decoder, and a mass storage 

55 for storing television signals, which could be a con- 
ventional video cassette recorder, a large direct ac- 
cess storage device, or other data storage. 

A television viewer inputs to the presentation sys- 
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tem one or more key words to be used as search para- 
meters. These may be input simply as an unordered 
set of words, or logical relations such as AND", "OR", 
etc. may be specified as known in the art of computer 
database searching. The computer stores these key 5 
words and their required logical relationships. The 
viewer also directs the presentation system to moni- 
tor specified channels at specified times. 

The television receiver/tuner receives television 
signals for the specified channel or channels. The 10 
closed captioning portion of the television signal is ex- 
tracted by the closed captioning text decoder and 
converted to a digital stream of textual data. The com- 
puter then scans the closed captioning textual data 
stream extracted from the video sig nal for occurrenc- 15 
es of words matching its search parameters. 

In a scan mode of operation, scanning is done 
while the television signal is being received. When a 
match is found, the corresponding segment of the tel- 
evision signal (video and audio portions) is saved in 20 
the mass storage for later viewing. In other modes of 
operation, the system scans the closed captioning 
stream of a pre-recorded signal for a match of the 
search parameters. 

What is done with a segment of interest once it 25 
has been located will depend upon the desires of the 
viewer. The viewer may, for example, specify the size 
of a segment, which could be a relatively short inter- 
val (e.g., 1 5 seconds) before and after the occurrence 
of the search term, as might be useful for a news 30 
broadcast, or could be a much longer interval such as 
an entire 1/2 hour television program. The viewer may 
wish to watch any segments which were found, or 
may wish to edit and save segments for later viewing. 
The viewer may wish to edit out small segments of a 35 
larger presentation, e.g., where the viewer finds such 
segments offensive. The system may display to the 
user all or part of the extracted textual data from the 
segments, and permit the user to select those seg- 
ments to be viewed and/or edited. 40 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of the major components 
of a video presentation system according to the 45 
preferred embodiment of this invention; 
Fig. 2 shows in greater detail the major compo- 
nents of the computer portion of the video pre- 
sentation system according to the preferred em- 
bodiment; 50 
Fig. 3 shows in greater detail the structure of the 
video segment buffer according to the preferred 
embodiment; 

Figs. 4A, 4B, 4C, 4D, and 4E show the format of 
scan profile data structures according to the pre- 55 
ferred and alternative embodiments; 
Fig. 5 is an example of how a complete set of pro- 
file data structures would be used to specify a 



typical scan for data of interest according to the 
preferred and alternative embodiments; 
Fig. 6 shows the high level flow of control of the 
control program according to the preferred em- 
bodiment; 

Fig. 7 shows the steps performed by control pro- 
gram 220 as part of the edit profile function ac- 
cording to the preferred embodiment; 
Figs. 8A and 8B show the steps required to scan 
a received signal for items of interest according to 
the preferred embodiment; 
Fig. 9 shows the steps required to determine 
whether a sequence of words in a log of closed 
captioning text matches the search parameters 
specified by the profile data structures according 
to the preferred embodiment; 
Fig. 10 shows the steps required to search and 
edit a signal according to the preferred embodi- 
ment 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

Figure 1 shows a block diagram of the major com- 
ponents of a video presentation system according to 
the preferred embodiment of the present invention. 
Video presentation system 100 comprises digitally 
controlled receiver/tuner 101, signal splitter 102, 
closed captioning text decoder 103, general purpose 
digital computer system 1 04, mass storage 1 05, video 
display 106 and audio speaker 107. Receiver/tuner 
101 receives an analog television signal from an ex- 
ternal source, such as an antenna for receiving at- 
mospheric television transmissions, a coaxial trans- 
mission cable from a cable television network, or a 
video cassette recorder. Receiver/tuner 101 receives 
commands to tune in one or more channels from com- 
puter 104 via control line 111. Receiver/tuner 101 de- 
modulates the signal received on a specified channel 
and outputs the demodulated signal on line 112 to 
splitter 102. The signal is fed simultaneously from 
splitter 102 into computer 104 and closed captioning 
text decoder 103 via lines 113,114. Closed captioning 
decoder 103 extracts a stream of textual data from 
the video signal. This extracted stream is forwarded 
to computer 104 in digital format on serial digital line 
115. Optional speech recognition unit 130 may also 
be used to provide additional textual data as ex- 
plained with reference to an alternative embodiment. 

Closed captioning decoder 1 03 is preferably a de- 
coder constructed to convert information contained in 
line 21 of the vertical blanking interval of a television 
signal to a text stream. Normally, this information is 
a modified representation of spoken text being car- 
ried in the audio portion of the television signal, and 
is intended for use by hearing impaired persons. 
Standards for such devices are specified in 47 C.F.R. 
15.119, herein incorporated by reference. Additional 
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information relating to dosed captioning decoders is 
contained in Lentz et al. ( Television Captioning for 
the Deaf Signal and Display Specifications" (Public 
Broadcasting Service, 1980), incorporated herein by 
reference as additional background information. s 

In the preferred embodiment, receiver/tuner 101 
is an IBM PS/2 TV. Computer 104 is an IBM PS/2 per- 
sonal computer. It should be understood that other 
hardware performing similar function could be used 
in place of that specified here, in particular any of va- 10 
rious commercially available video function cards 
which plug into available slots of a personal computer. 

Fig. 2 shows in greater detail the major compo- 
nents of computer portion 104 of video presentation 
system 100. Computer 104 comprises central proc- 15 
essing unit (CPU) 202, random access memory 203, 
mouse port controller 204, keyboard port controller 
205, serial communications controller 206, digital vid- 
eo capture 207, graphic display card 208, audio driver 
209 and storage controller 210, all coupled to bidirec- 20 
tional system communications bus 201. Bus 201 fa- 
cilitates communications, i.e. transfer of data, among 
the various components of computer 104. 

CPU 202 is a general purpose programmable 
processor such an an Intel 80486 processor common- 25 
ly used in personal computers. Memory 203 is a ran- 
dom access memory sufficiently large to hold the 
necessary programming and buffer data. While mem- 
ory 203 is shown as a single entity, it should be un- 
derstood that memory 203 may in fact comprise a 30 
plurality of modules, and that memory may exist at 
multiple levels, from high-speed registers and caches 
to lower speed but larger DRAM chips. As shown in 
Fig. 2, memory 203 contains control program 220, 
data structures 221 , and buffer 222. Control program 35 
comprises a plurality of machine instructions which 
execute on processor 202 to perform video selection, 
scanning and editing functions as described more 
fully herein. Data structures 221 comprise data which 
direct and customize the operation of control program 40 
220, such as user time and channel selections, 
search parameters, locations of video segments of in- 
terest, etc. Buffer 222 is used for temporary storage 
of video signals or closed captioning data during 
scanning and editing by control program 220. 45 

Mouse port controller 204 and keyboard port con- 
troller 205 facilitate input of data by the viewer from 
either a mouse or keyboard. In the preferred embodi- 
ment, keyboard 120 attaches to PS/2 TV 101, and 
keyboard signals pass through it. However, it is also so 
possible to couple a keyboard directly to keyboard 
port 205. Controllers 204 and 205 are illustrative of 
the type of input controller commonly used for per- 
sonal computers, it being understood that it would be 
possible to use other forms of input devices, such as 55 
joysticks, trackballs, infra-red hand-held remote con- 
trol units, or even speech recognition devices. 

Serial communications controller 206 comprises 



a conventional universal asynchronous receiv- 
er/transmitter (UART), for handling serial communi- 
cations with closed captioning decoder 1 03 over ser- 
ial digital line 115. While controller 206 is capable of 
bidirectional communications, in general it only re- 
ceives digital data from decoder 103. However, it may 
be used to transmit commands to decoder 103, e.g., 
to re-try a communication. 

Digital video capture unit 207 is a circuit card and 
components which receive analog video and audio 
signals over line 114 and convert these signals to a 
digital format suitable for storage and manipulation by 
digital computer 104. In the preferred embodiment, 
digital video capture unit 207 is an IBM ActionMedia 
II Display Adapter with ActionMedia II capture option. 

Graphic display card 208 transmits outgoing vid- 
eo signals to receiver/tuner 101 over line 116, and ul- 
timately to video display screen 1 06. Display card 208 
receives digital signals on bus 201 and converts 
these to an RGB analog format or composite video 
analog format, suitable for receiving by display screen 
106. Audio driver 209 simultaneously transmits the 
audio component of the television signal to speaker 
107. 

Storage controller 210 communicates with mass 
storage device 105. In the preferred embodiment, 
mass storage device 105 is an IBM PS/2 External 
Disk Array Subsystem. Such a device comprises a 
plurality of rotating magnetic disk drive storage devic- 
es, which are controlled by a single controller to act 
as a single large storage entity. Mass storage device 
1 05 functions primarily as a buffer for storage of vid- 
eo and audio signals which have been digitized by 
capture unit 207. Due to the large volume of storage 
such digitized signals consume, a large external stor- 
age device is preferably employed. While only one 
such unit is shown, it should be understood that there 
may in fact be multiple mass storage devices, which 
may be internal or external to computer 104. Typical- 
ly, computer 104 will include an internal disk drive of 
smaller capacity than the external disk drive array 
subsystem 105, on which may be stored, e.g., pro- 
grams required for operation of computer 104 in ac- 
cordance with the present invention or other data. It 
should also be understood that other forms of mass 
storage for video signals may be employed. In partic- 
ular, it would be possible to store analog video signals 
on one or more conventional video cassette recor- 
ders. 

Fig. 3 shows in greater detail the structure of buf- 
fer 222. Buffer 222 is part of a larger general purpose 
RAM 203. A portion of RAM 203 is allocated to buffer 
222 upon initializing a certain functions as described 
more fully herein. Buffer 222 is allocated in a plurality 
of segments 301-312, which may or may not be con- 
tiguous in RAM 203. Each buffer segment is capable 
of holding a television signal segment from a short 
time period in its respective data field 323. Each buf- 
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fer segment 301-312 also contains a respective poin- 
ter field 321, containing a pointer to the beginning of 
the next segment, and a respective save field 322, 
containing a flag indicating whether the segment 
should be saved to storage. Segment pointer 325 in s 
data 221 points to current segment 305. Buffer 222 is 
preferably arranged as a circular buffer. Segment 
pointer 325 is Incremented" periodically as required 
to point to the next segment. After the last segment is 
reached, the pointer is Incremented" to point to the 10 
first segment again. As used herein, "incremented" 
refers to the fact that the pointer is set to the next seg- 
ment in the circular buffer; the actual numerical ad- 
dress in the segment pointer may be increased or de- 
creased. While Fig. 3 shows a buffer having 12 seg- 15 
ments, it should be understood that the number and 
size of the segments may vary, and in particular that 
control program 220 may dynamically vary the num- 
ber and size of the segments. The design and oper- 
ation of such circular buffers is known in the art of 20 
computer programming. 

Figs. 4A, 4B, 4C, 4D and 4E show the format of 
the scan profile data structure which is used to record 
what type of television information the viewer wishes 
to identify, and to govern the operation of control pro- 25 
gram 220. Each profile record includes one profile 
control block 401 as shown in Fig. 4A. Profile control 
block 401 is the main profile record, and contains 
pointers which point to subsidiary records. Profile 
control block 401 comprises title field 402, which con- 30 
tains a short title of the profile for use in viewer iden- 
tification. Description field 403 contains a longer tex- 
tual description of the subject of the profile. Interval 
field 404 determines the size of an interval to be 
saved and recorded. It specifies the number of sec- 35 
onds of television signal to be saved and recorded be- 
fore and after encountering a match of the search 
parameters. Next profile pointer field 405 and last 
profile pointer field 406 contain pointers to the next 
and previous profile control blocks, respectively. It is 40 
expected that the video presentation system of the 
present invention may store multiple profiles, and that 
these would be stored as one or more circular linked 
lists of variable length, as are known in the art of com- 
puter programming. Time pointer field 407 contains a 45 
pointer to the first time control block 410 to which the 
profile applies. Time control blocks 410 are main- 
tained as a linked list of variable length, allowing a sin- 
gle profile to specify multiple time periods during 
which channels should be scanned for television of in- so 
terest Keyword pointer 408 contains a pointer to the 
first OR-ed key word control block 430. Keywords are 
also maintained as a variable length linked list, per- 
mitting multiple keywords and combinations of AND- 
OR logic, as illustrated below in respect to Fig. 5. 55 

The format of time block 410 is shown in Fig. 4B. 
Each time block 410 specifies a time period during 
which applicable channels should be scanned for 



items of interest Date or day field 411 specifies a 
date or day of the week for the start of the scanning 
period. This may be specified either as a single date, 
in which case the scan will be performed only once, 
or as a day of the week, in which case the scan will 
be performed every week at the same time. Date or 
day field 411 may also contain a special code desig- 
nating Monday through Friday" or other common 
combination of days, indicating that the scan is to be 
performed on all such days. Start time field 412 spe- 
cifies the time of day at which the scan is to begin. 
Time end field 413 specifies the time at which the 
scan ends. Next time pointer 414 contains a pointer 
to the next time control block 410 in the list of time 
control blocks for the applicable profile. A null value 
indicates that the end of the list has been reached. 
Channel pointer 415 contains a pointer to the list of 
channels to be scanned. 

The format of channel block 420 is shown in Fig. 
4C. Channel block 420 is used only if multiple channel 
scanning is supported. Where the system is capable 
of scanning only one channel at a time, there is no 
need for a pointer to a list of channels, and field 415 
of time block 410 may instead contain simply the sin- 
gle channel to be scanned. Where multiple channels 
may be scanned, provision is made for a variable 
length list of channels. Channel block 420 contains 
channel field 421 , which identifies a channel to be 
scanned, and next channel pointer field 422, which 
contains a pointer to the next channel control block on 
the list. The final channel block on the list contains a 
null pointer in field 422. 

The formats of OR-ed keyword block 430 and 
AND-ed keyword block 440 are shown in Figs. 4D and 
4E respectively. These blocks effectively specify the 
search parameters to be used when scanning for tel- 
evision data of interest. In accordance with the prefer- 
red embodiment, a user may specify one or more dis- 
junctions of conjunctions of keywords. I.e., a user 
may specify a search of the form (WORD1 and 
WORD2) or (WORD3 and WORD4 and WORDS) or 
(WORD6) or... The number of disjunctions is variable, 
as is the number of conjunctions which make up each 
disjunction. OR-ed keyword block 430 comprises 
next OR-ed pointer field 431 , which points to the next 
OR-ed keyword block of the disjunctive string; key- 
word 432 which contains a keyword to be matched; 
and next AND-ed pointer field 433, which contains a 
pointer to an AND-ed keyword block 440 which is part 
of the conjunctive string. ANDed keyword block 440 
comprises number words within field 441 , which spe- 
cifies a maximum distance in terms of number of 
words between words in a conjunctive string. I.e., the 
words of a conjunctive string must be within a speci- 
fied proximity of each other to create a true match. 
ANDed keyword block 440 further comprises key- 
word field 442, which specifies a keyword, and next 
ANDed pointer 443, which points to the next ANDed 
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keyword block 440 of the string. Nulls in pointer fields 
431 ,433,443 indicate the last word of the string. 

Fig. 5 is an example of how a complete set of pro- 
file data structures 401, 410, 420, 430, 440 would be 
used to specify a typical scan for data of interest. In 
this example, the profile is established to scan for 
news about the Minnesota Twins baseball team. It is 
desired to scan local channels 3 and 10 during the 
periods of normal local newscasts on weeknights, 
and to scan major network channels 4, 5 and 6 all 
weekend for items about the Minnesota Twins. 

In the example of Fig. 5, reference number 501 
represents a profile control block having the same 
structure as profile control block 401 shown in Fig. 
4A. Reference numbers 510, 513 and 516 each rep- 
resent a time block having the same structure as time 
block 410 shown in Fig. 4B. Reference numbers 511 , 
512, 514, 515, 517, 518 and 519 each represent a 
channel block having the same structure as channel 
block 420 shown in Fig. 4C. Reference numbers 520 
and 522 each represent an OR-ed keyword block hav- 
ing the same structure as OR-ed keyword block 430 
shown in Fig. 4D. Reference numbers 521 and 523 
each represent an AND-ed keyword block having the 
same structure as AND-ed keyword block 440 shown 
in Fig. 4E. 

As noted earlier, the complete set of profile data 
for this scan contains one profile control block 501 . Ti- 
tle field 402 of block 501 contains the title Twins", 
and description field contains the string *MN Twins 
Baseball News". These fields are useful in identifying 
the nature of the profile. Next and Last profile pointer 
fields 405,406 point to additional profile control 
blocks (not shown) having the same structure as pro- 
file control block 401 , which are used tof ind unrelated 
items of interest 

Time block pointer field 407 points to the first 
time block 510 of a string of time blocks. In this ex- 
ample, the first time block 510 is used to specify the 
time period from 5:00pm to 6:30pm on weekdays. 
This time period corresponds to the time for typical lo- 
cal newscasts. Channel pointer field 41 5 of block 510 
points to channel block 511. Channel field 421 of 
channel block 51 1 contains a channel (channel 10) to 
be scanned during the time period specified by time 
block 510. Next channel pointer field 422 points to 
channel block 512. This channel block contains a dif- 
ferent channel (channel 3) to be scanned during the 
same time period. Next channel pointer field 422 of 
block 512 contains a null pointer, indicating the end of 
the list of channels to be scanned at this time. Thus, 
blocks 510, 511 and 512 together specify that chan- 
nels 1 0 and 3 should be scanned every weekday from 
5:00pm to 6:30pm. 

Next time pointer field 414 of time block 51 3 spe- 
cifies another time period to be scanned. As in the 
case of time block 510 described above, time block 
513 points to channel block 514, which in turn points 



to block 51 5. These collectively specify that channels 
10 and 3 are to be scanned every weekday from 
10:00pm to 10:30pm. Although the strings of blocks 
starting at 511 and 514 are shown separately in Fig. 

5 5 for clarity, it should be understood that they may in 
fact be a single string of channel blocks, which is 
pointed to by both time blocks 510 and 513. 

Next time pointer field 414 of time block 513 
points to time block 516, which specifies still another 

10 time period to be scanned. Block 51 6 specifies a scan 
on Saturday and Sunday, from 8:00am to 11:00pm. 
Next time pointer field 414 of block 516 is null, indi- 
cating the end of the string of time blocks 510, 513, 
and 516. Channel pointerfield415of block 51 6 points 

15 to channel block 517, which specifies channel 4. 
Channel block 517 points to channel block 518, which 
in turn points to channel block 519, specifying a list 
of three channels, channels 4, 5 and 6, to be scanned 
during the time period specified by time block 516. 

20 Keyword pointer field 408 of profile control block 

501 points to the first OR-ed keyword block 520. OR- 
ed keyword block 520 contains the keyword "Minne- 
sota" in keyword field 432, and contains a pointer to 
AND-ed keyword block 521 in next AND-ed ptr field 

25 433. AND-ed keyword block 521 contains the key- 
word Twins" in keyword field 442, and contains the 
number 2 in field 441, specifying the maximum dis- 
tance of the keywords. Next ANDed pointer field 443 
of block 521 is null, indicating the end of the string 

30 of ANDed keywords. Blocks 520 and 521 together 
specify a search for the keywords "Minnesota "and 
Twins", and specify that these words must be within 
two words of each other in a text string. Thus, the text 
"Siamese twins were born at Lake Wobegone, Minne- 

35 sota" would not satisfy the search parameters speci- 
fied by blocks 520 and 521 because the words twins" 
and Minnesota" are more than two words apart in the 
string, whereas The Minnesota Twins lost to Kansas 
City last night" would satisfy the search parameters. 

40 Typically, closed captioning text is in all capital 

letters; the sample text above is shown in mixed case 
for ease of understanding. However, even if text is 
presented in mixed case, the search parameters are 
preferably not case sensitive, i.e., search words 

45 match regardless of whether the letters are upper or 
lower case. 

Next OR-ed pointer field 431 of block 520 points 
to block 522, which specifies another string of key- 
words. Block 522 contains the keyword Twins" in 

so keyword field 432, and a pointer to AND-ed keyword 
block 523 in field 433. AND-ed keyword block 523 
contains the keyword "Baseball" in field 442, and the 
keyword distance 30 in field 441 . A null pointer in field 
443 of block 523 indicates the end of the conjunctive 

55 string. Blocks 522 and 523 collectively specify a 
search for the keywords Twins" and "Baseball", 
which must be located within 30 words of each other 
in a text string. Next OR-ed pointer field 431 of block 
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522 is null, indicating the end of the string of OR-ed 
keyword blocks (disjunctive string). In this example, 
two separate and independent (disjunctive) condi- 
tions are specified, the first by blocks 520-1 , the sec- 
ond by blocks 522-3. If a decoded closed captioned s 
text string satisfies either condition, the search para- 
meters are said to be satisfied and appropriate action 
will be taken to save or otherwise deal with the video 
interval in which the string satisfying the search para- 
meters was found, as more fully described below. 10 

The operation of the present invention in the pre- 
ferred embodiment will now be described. Control 
program 220 is the heart of the operation, and con- 
trols computer 104 and other hardware shown in Fig. 
1. Control program 220 comprises four basic func- 15 
tions: a set profile function, a capture function, a scan 
function, and a review/edit function. 

In order to perform certain functions, control pro- 
gram 220 requires interactive input of the viewer. Vid- 
eo presentation system 100 preferably displays 20 
prompts, lists of choices, and other required informa- 
tion on display 106. Information in text form required 
for viewer selections may be overlayed on any video 
picture being displayed on display 106, permitting the 
viewer to interactively input information while a tele- 25 
vision program is simultaneously being displayed on 
display 1 06. Apparatus for overlaying such textual in- 
formation is known in the art. Alternatively, it would be 
possible to display such textual information on dis- 
play 106 by itself, without any other video picture, or 30 
in a window of display 106 while a television program 
is being shown on the remainder of the display, or on 
a separate special purpose display, such as a small 
light emitting diode (LED) display. 

Fig. 6 shows the high level flow of control of con- 35 
trol program 220. Upon power first being applied to 
video presentation system 1 00, control program 220 
enters idle mode at block 601. Control program 220 
waits in idle mode until the user demands attention, 
as for example, by pressing an appropriate function 40 
key on keyboard 120, or a button on mouse 121 (block 
602). Control program responds by displaying a menu 
of available choices on display 106 or a special pur- 
pose display, as explained above, at block 603. The 
available choices are to edit a profile (block 610), to 45 
scan incoming television signals in accordance with a 
profile (block 611), to search and edit a stored video 
signal (block 612), and to capture and save incoming 
television signals for later viewing and/or editing 
(blocks 620-23). The user may also choose to return so 
to idle mode (block 613). The user selects one of the 
available choices, and upon completion of the selec- 
tion, control program 220 either returns to idle mode 
or to the menu of choices, as illustrated. The steps re- 
quired to perform the edit profile, scan, and 55 
search/edit video functions (blocks 610-612) are ex- 
plained in greater detail below with reference to Figs. 
7-10. 



To perform the capture and save function, the 
video presentation system must be instructed as to 
the time interval and channel(s) to be captured and 
saved. When this option is selected, control program 
presents the viewer with a list of available profiles to 
select at block 620. These profiles specify channel(s) 
and time(s) for capture of the signal. The viewer may 
optionally specify a channel and start and stop times, 
without creating a profile. Upon receiving the viewer's 
selection, control program 220 waits at block 621 un- 
til the specified start time for video capture. At the 
start time, control program 220 captures the video 
and text. I.e., it commands digital tuner 101 to tune 
the appropriate channel, and instructs video capture 
unit 207 to receive and digitize the analog signal from 
tuner 101 and send it directly to storage controller 210 
via bus 201 to be stored in storage 105 (block 622). 
At the same time, decoder 103 decodes closed cap- 
tioning data, which is received by serial UART 206 
and also stored directly in storage 1 05 (block 623). At 
the end of the specified capture period, control pro- 
gram 220 returns to idle mode. The television signal 
thus digitized and stored can be later edited using the 
search/edit function, or simply played back. 

Fig. 7 shows the steps performed by control pro- 
gram 220 as part of the Edit Profile function. The Edit 
Profile function allows a viewer to create or alter a 
profile data structure. The profile data structure spe- 
cifies such things as channels and times to be moni- 
tored, size of capture windows, key words used and 
search parameters and logical relationships between 
these key words, etc., as explained above. When a 
viewer selects the set profile function, e.g. by key- 
board 120 or mouse input device 121, monitor pro- 
gram 220 displays a menu of available choices, which 
include to create a new profile from scratch, to edit an 
existing profile, or to copy an existing profile as a tem- 
plate for creating a new profile. If the user elects to 
create a new profile from scratch, control program 
220 prompts the user to enter the new profile name 
at block 701. It then creates the new profile data 
structures explained above and depicted in Figs. 4A- 
4E, and enters default values in the appropriate 
fields, at block 702. If the user elects to edit an exist- 
ing profile, control program 220 displays a list of ex- 
isting profiles and receives the user's selection at 
step 703. If the user elects to copy an existing profile 
as a template, control program 220 displays the list of 
existing profiles and receives the user's selection at 
step 704. It then creates new profile data structures 
as depicted in Figs. 4A-4E, and copies the values 
from the selected profile into the new profile data 
structures, at step 705. Program 220 then prompts 
the user to enter the name of the new profile, in order 
to distinguish it from the profile from which it was cop- 
ied, at step 706. 

Upon completing the initial creation, selection or 
copying of a profile as explained above, control pro- 
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gram 220 displays the profile values at step 71 0. The 
user may then select any of various editing options. 
The user may add an element (block 711), as for ex- 
ample adding a time period to be scanned or adding 
a search word. The user may delete such an element 5 
(block 712). The user may change such an element 
(block 713). Each time the user adds, deletes or 
changes an element, control program returns to block 
710 to display the altered profile. The user may also 
delete an entire profile, which would delete all asso- 10 
dated data structures depicted in Figs. 4A-4E (block 
714). When done editing, the user chooses to end the 
editing function (block 715). 

In accordance with the preferred embodiment, 
the video presentation system can operate in auto- is 
matic scan mode to automatically find and record tel- 
evision signals of interest without a human operator's 
attendance during signal reception. Figs. 8Aand 8B 
illustrate the operation of the presentation system 
while in automatic scan mode. More particularly, 20 
Figs. 8Aand 8B illustrate the steps performed by con- 
trol program 220 executing on CPU 202 of computer 
104. 

Upon entering automatic scan mode, control pro- 
gram 220 performs necessary initialization of data 25 
221 and buffer 222 at step 801 . Because buffer 222 
is part of a larger general purpose RAM 203, initiali- 
zation requires allocation of a portion of RAM 203 to 
buffer 222. Buffer 222 will be allocated in a plurality 
of segments 301-31 2, each buffer segment holding a 30 
television signal segment from a short time period. 
This time period is *fehort" in relation to human per- 
ceptions and viewing time. In relation to the operation 
of a digital computer rt must be sufficiently long to per- 
mit computer 104 to perform the steps shown in Figs. 35 
8A and 8B. This time period may, for example, be in 
the vicinity of 2 seconds. Segment pointer 325 is ini- 
tialized to a first of such segments. 

For monitoring and storing television signals in 
real time in accordance with the preferred embodi- 40 
ment, the number and size of segments in buffer 222 
must be sufficiently large that an interval of the spe- 
cified size can be captured. For example, if it is de- 
sired to capture an interval of television signal begin- 
ning 15 seconds before closed captioning data 45 
matching a set of search parameters is encountered, 
then buffer 222 must be sufficiently large to hold 15 
seconds of digitized television signal. If each segment 
holds 2 seconds of televised signal, then buffer 222 
should hold a minimum of 9 segments. One extra seg- so 
ment is needed to account for the fact that control pro- 
gram 220 is analyzing a segment previous to the one 
being currently recorded. Control program 220 allo- 
cates a buffer of appropriate size based on scanning 
parameters. 55 

Alternatively, in certain applications it may be de- 
sirable to record much larger intervals. For example, 
a viewer may wish to save an entire half- hour pro 
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gram if a set of search parameters is satisfied any- 
where within the program. The set of search parame- 
ters may even require a count of frequency of occur- 
rence of some condition, so that the program should 
be saved only if the condition is met a specified num- 
ber of times. In this case, computer 104 may be un- 
able to determine that the television program should 
be saved until the transmission is nearly finished. Be- 
cause this may require buffers of extremely large 
size, computer 1 04 could alternatively save the entire 
television program in storage 105 while it is being 
transmitted, analyze the closed captioning data as 
described herein, and delete the program from stor- 
age 105 after transmission of the program is com- 
plete if the specified conditions are not met 

After the buffer and data initialization functions 
have been performed, control program 220 waits in 
a loop for the start time at step 802. The start time, 
i.e., the time to begin receiving and monitoring a tel- 
evision signal, is specified in a profile data structure. 
Multiple profile data structures may exist, each spec- 
ifying a certain start time. While waiting at step 802, 
the control program repeatedly compares the various 
start times in the profile data structures with an inter- 
nal clock. When the internal clock reaches the first 
start time, the control program exits the wait loop. If 
multiple profile data structures exist, control program 
220 will periodically check whether the respective 
start time of each has arrived and respond appro- 
priately; for brevity of illustration, this is not shown in 
Fig. 8A. 

To start monitoring and capture of video signals, 
control program 220 sends a channel select com- 
mand to receiver/tuner 101 over control/keyboard 
line 111 at step 803. The channel select command 
specifies a television channel to be monitored, the 
channel being the one specified in the respective pro- 
file data structure containing the start time which has 
been triggered. Digital receiver/tuner 1 01 responds to 
the channel select command by tuning the specified 
channel and transmitting the tuned, demodulated tel- 
evision signal on analog signal line 112. At the same 
time, control program 220 sends a command to video 
capture unit 207 to begin receiving video signals over 
line 114, and to write the digitized video signals into 
the current segment in buffer 222 (as being pointed 
to by current segment pointer 325). 

If the profile specifies that multiple channels are 
to be scanned, control program 220 will command tu- 
ner 101 to cycle through the channels of interest, thus 
obtaining closed captioning text from multiple chan- 
nels. Because the amount of closed captioning text is 
relatively small, and is intended to be displayed on a 
television for a time interval which is relatively long 
compared to the time required for digital hardware to 
decode the closed captioning signal, it is possible for 
a single tuner to obtain samples from multiple chan- 
nels by cycling through the channels and decoding 
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the closed captioning signal. However, in this case it 
is not possible for the hardware shown in Fig. 1 to 
save the video signal from multiple channels simulta- 
neously. In other words, multiple channel scanning is 
possible, but not multiple channel saving of video. If 
an item of interest is found, video from that channel 
can be saved upon detection of the match with the 
search parameters specified in the profile. Depend- 
ing on the application, this small delay to start saving 
the video segment may be acceptable to the viewer. 
In the multiple channel hardware alternative embodi- 
ment described more fully below, control program 
220 will command the appropriate tuning hardware to 
monitor the specified multiple channels, enabling 
multiple channels to be saved. 

Computer 104 simultaneously captures the cur- 
rent video segment and analyzes the previous video 
segment. These concurrent processes are illustrated 
in Fig. 8A by the divergent lines of flow going to blocks 
804 and 810. As represented by block 804, video cap- 
ture unit 207 receives the demodulated analog tele- 
vision signal from tuner 101 and splitter 102 over line 
114. Capture unit 207 converts this signal to digital 
form, and transmits the digitized signal data over bus 
201 to buffer 222. Data is preferably transferred in a 
direct memory access (DMA) operation requiring no 
intervention by CPU 202, as is known in the art of 
computer architecture. Because video capture unit 
207 receives television signals in real time, it can not 
write data to buffer 222 any faster than the data is re- 
ceived over line 114. A2-second television segment 
requires 2 seconds for capture unit 207 to receive and 
write to buffer 222. While this is a relatively short in- 
terval in terms of human viewing time, it is more than 
long enough for a computer to perform the steps 
shown in blocks 810-828. These steps are performed 
by control program 220 executing on CPU 202 while 
video capture unit 207 captures the signal. 

As depicted by block 810, control program 220 
first fetches the closed captioned digital text stream 
associated with the previous segment of video. This 
is the text stream extracted from the closed caption- 
ing portion of the television signal by closed caption- 
ing text decoder 103. This text stream is transmitted 
by decoder 103 to computer 104 over serial line 115. 
Serial communications controller 206 handles com- 
munication of data over line 115. Received text is 
saved in a log record (not shown) in storage 105, hav- 
ing timestamps or other means to correlate the re- 
ceived text to a particular video interval from which it 
came. Additionally, the most recent text will be re- 
quired for analysis. Because the amount of closed 
captioning text contained in a typical video interval is 
relatively small, the recent text required for analysis 
may be stored in a small buffer (not shown) in control- 
ler 206, or in a section of memory 203. Where multiple 
channels are being scanned, a separate text log and 
temporary buffer would be required for each channel 



being scanned. 

Control program 220 next analyzes the text 
stream(s) with respect to the active prof ile(s) at step 
811. Step 811 is broken out in greater detail in Fig. 8B, 

5 as described below. 

Referring to Fig. 8B, control program 220 com- 
pares a logged in text stream against the parameters 
of a profile to determine whether the text stream 
matches search criteria specified in the applicable 

10 profile at step 820. In the preferred embodiment the 
search criteria comprises one or more keywords con- 
nected by logical relationships. The steps required to 
form a determination as to whether a match exists are 
represented in Fig. 9. 

15 Referring to Fig. 9, a determination whether a 

match exists is made by scanning each keyword in an 
ORed keyword block 430 or ANDed keyword block 
440 for a match against each word in the log of closed 
captioning text Words in the log of closed captioning 

20 text are selected serially for matching at step 901. 
Control program 220 accesses profile control block 
401 to obtain the keyword block at the head of the first 
conjunctive string (the various conjunctive strings of 
keywords being ORed together to form disjunctive 

25 strings) at step 902. Referring to the example of Fig. 
5, ORed keyword block 520 is the first keyword block 
obtained. Control program 220 obtains the keyword 
from the appropriate keyword block for matching at 
step 903. In this example, keyword "Minnesota" is ob- 

30 tained the first time through the loop. 

At step 904, control program compares the key- 
word obtained from the appropriate keyword block 
with the current word in the log of closed captioning 
text to determine whether the two words match. A - 

35 "hrtatch " does not necessarily mean each character of 
the two words is identical. The keyword may option- 
ally contain wildcard characters to broaden the scope 
of matches. For example, a wildcard character such 
as m £ m may be used to indicate a variable number of 

40 letters of any kind. Such a wildcard character would 
normally be attached to a root word in order to match 
on the root and any variant of the root For example, 
a keyword of "truck£" would detect a match not only 
for a logged word of "truck", but also for "trucks", 

45 Trucked", Trucking", "trucker", etc. Use of such wild- 
card characters is known in the art of computer data- 
base searching. 

If the keyword and the current logged word being 
analyzed do not match, control program 220 then de- 

so termines whether there are any additional keywords 
in the conjunctive string at step 905. If so, control pro- 
gram 220 loops back to step 903 to get the next key- 
word in the string. In the example of Fig. 5, control 
program 220 would fetch the keyword "Twins" from 

55 ANDed keyword block 521 . When all the keywords of 
a single conjunctive string (e.g., blocks 520 and 521) 
have been compared to the current logged word, con- 
trol passes to block 906. 
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At step 906, control program 220 determines 
whether there are any more conjunctive strings of 
keywords to compare against the current logged 
word. If so the control program loops back to step 902 
to get the head of the next conjunctive string, and pro- s 
ceeds through blocks 903-905 to compare each key- 
word of the string to the current logged word. In the 
example of Fig. 5, blocks 522 and 523 constitute the 
next conjunctive string. 

When all conjunctive strings have been com- 10 
pared against the current logged keyword, control 
program proceeds to step 907 to determine whether 
any more words from the log of closed captioning text 
remain to be compared to the keywords. If so, the con- 
trol program loops back to step 901 to get the next is 
word in the log. When all words in the log have been 
analyzed without finding a match, control program 
220 returns with a determination that no match has 
been found. 

If, while traversing the keyword blocks, control 20 
program 220 determines at step 904 that any current 
logged word matches a keyword, it must then deter- 
mine whether all the keywords in the same conjunc- 
tive string are matched by some word in the log of 
closed captioning text (in order to satisfy the logical 25 
AND condition). The words in the closed captioning 
text must be within the proximity limit specified in 
proximity field 441 of ANDed keyword block 440. 
Control program 220 therefore obtains each keyword 
in turn from the conjunctive string (step 910), scans 30 
the P most recent words of the dosed captioning log 
to determine whether a match exists, where P is the 
proximity limit specified in field 441 (step 911), and 
determines whether all keywords in the conjunctive 
string have been compared to the log (step 912). If 35 
any keyword fails to match a word in the log at step 
911 , the conjunctive string as a whole does not match 
the log and control program proceeds to step 906 to 
determine whether any additional conjunctive strings 
remain to be analyzed. If all keywords in the conjunc- 40 
th/e string are compared to the log at step 911, and a 
match is found for each one, then the "N" branch from 
block 912 will be taken at the end of the string, and 
control program returns with a determination that a 
match of closed captioning text with search parame- 45 
ters has been found. 

If control program 220 following the logic depict- 
ed in Fig. 9 determines at block 820 that no match ex- 
ists, it then checks at step 821 whether a match flag 
is set on. The match flag indicates that a match has so 
been found in a previous recent segment, i.e., one 
within the limits of the interval specified in interval 
field 404 of profile control block 401 . If the match flag 
is off, there has been no match of the search parame- 
ters in any recent segment, and control proceeds to 55 
block 822 or 823. If the match flag is on, the segment 
being analyzed must be saved, and control proceeds 
to block 832. 



If control program 220 determines at block 820 
(following the logic depicted in Fig. 9) that a match ex- 
ists, it then sets a match timer to the interval length 
specified in interval field 404 of profile control block 
401 at step 830. The match timer is set regardless of 
whether the timer is currently active. The control pro- 
gram then checks the status of the match flag at step 
831 . If the match flag is off, no match has been found 
within the time specified by the interval. In this case, 
it is necessary to first save all earlier segments within 
the interval specified by interval field 304 which have 
not already been saved. Control program first sets 
the match flag on at step 836. It then scans all earlier 
segments currently in the buffer and recorded within 
the interval to identify which ones are not Imarked", 
and commands storage controller 21 0 to save all such 
segments in storage 105, at step 837. Control pro- 
gram 220 then marks all segments it has identified to 
be saved by setting save flag field 322 at step 838. 
The purpose of marking segments is so that they are 
not saved again. Control then proceeds to block 832. 

Storage controller 21 0 has direct memory access 
capability. In response to a command from control 
program 220 to store a segment issued at step 837, 
controller 210 independently and asynchronously re- 
trieves the specified segment from buffer 222 and 
stores it in storage 105. Once control program 220 
has issued the command to storage controller 21 0 to 
store the segment, control program 220 need no lon- 
ger supervise the operation, and may continue proc- 
essing on the assumption that the segment will be 
stored in parallel with further processing in CPU 202. 

If at block 831, the match flag is on, control pro- 
gram 220 has detected a second match within the in- 
terval. In this case, it is presumed that the control pro- 
gram has already caused storage controller 210 to 
save those segments earlier than the segment being 
analyzed ( ■previous" segment). Accordingly, steps 
836-838 are not performed, at control flows directly 
to block 832. 

Block 832 is reached because the match flag is 
on, indicating that the segment being analyzed, i.e. 
the previous segment, should be saved to storage. In 
this case, control program 220 issues the appropriate 
command to controller 210 to save the previous seg- 
ment at step 832, and then sets save flag 322 to mark 
the previous segment at step 833 so that it will not be 
saved again. 

At block 834, control program 220 checks to see 
whether the match timer has timed out, indicating that 
the time specified in interval field 404 has elapsed 
since the last match of the search parameters was en- 
countered. If the timer has timed out, the match flag 
is reset (set to 'off) at step 835, so that no further 
segments will be saved unless another match of the 
search parameters is encountered. Control then pro- 
ceeds to block 822 or 823. 

Block 822 is applicable only if multiple channel 
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scanning is in effect. In this case, control program 220 
will return to block 820 to perform steps 820-1 and 
830-8 for each channel specified in the profile, if 
more than one channel is being monitored. A separate 
match flag and timer will exist for each channel being 
monitored. When the text streams from all channels 
have been analyzed, control proceeds to block 823. 

At block 823, control program 220 will return to 
block 820 to repeat steps 820-2 and 830-8 for each 
separate profile. When all profiles have been ana- 
lyzed, control returns to block 813 of Fig. 8A. 

At block 813, control program 220 checks an in- 
ternal system clock to determine whether the time 
specified in time end field 413 has passed. This time 
indicates the end of the scanning period. If the time 
has passed, control program 220 exits scanning 
mode and returns to the main control loop shown in 
Fig. 6. Where more than one profile is active and be- 
ing used in the scan, scanning will end for each profile 
independently, so that control program does not re- 
turn from scan mode until the last time specified in 
any profile. If the time has not yet passed, control pro- 
gram "increments" the segment pointer at step 814. 
It then issues a command to video capture unit 207 
to place the next segment (after the current segment) 
of digitized video in buffer 222 at the location pointed 
to by the segment pointer at step 81 5. Control pro- 
gram then waits at step 816 until video capture unit 
207 is done decoding and storing the current segment 
in the buffer. When capture unit 207 is done with the 
current segment, control program loops back to block 
810 and begins processing the segment which cap- 
ture unit 207 has just decoded and placed in the buf- 
fer. 

Typically, automatic scan mode as depicted in 
Figs. 8A and 8B will be used to scan and record short 
segments of interest for later viewing. When the view- 
er is ready to view the saved segments, he merely 
commands video presentation system 1 00 to play the 
segments on television monitor 1 06. To play the stor- 
ed segments, presentation system 1 00 retrieves the 
saved segments from storage 1 05, converts the stor- 
ed digitized signal to an analog television signal with 
display driver 208, and transmits the analog televi- 
sion signal via signal line 116 to tuner 101 and ulti- 
mately to monitor 106. During playback, the audio 
portion of the stored digitized television signal is con- 
verted to analog by audio driver 209 and output on 
speaker 1 07. 

Video presentation system 100 of the present in- 
vention is further capable of operating in a search/edit 
mode, as depicted in Fig. 10. Unlike automatic scan 
mode, search/edit mode is intended as an aid to man- 
ual editing of the signal. Specifically, search/edit 
mode uses techniques similar to those used in auto- 
matic scan mode to scan a television program and lo- 
cate segments that the viewer may wish to edit. Once 
these segments have been located, the viewer may 
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edit the television program by using any of various 
conventional editing techniques. 

When operating in search/edit mode, video pre- 
sentation system 100 must first extract and build 

5 strings of text decoded from the closed captioning 
stream at step 1 001 . This may be done in a variety of 
ways. In one option, the closed captioning stream can 
be decoded and saved as strings of text simultane- 
ously with the reception and storage of a broadcast 

10 television transmission. The stored television pro- 
gram is later edited in search/edit mode. In a second 
option, a recorded program can be scanned from be- 
ginning to end to extract and decode the closed cap- 
tioning stream, building the strings of text This step 

15 could be performed by the video presentation system 
without the need for the viewer to be monitoring the 
process, so that even if it requires playing the entire 
program from start to finish, the viewer is freed from 
the tedious task of monitoring. This second option 

20 would be useful, e.g., for scanning a pre-recorded vid- 
eocassette. In a third option, the extracting and de- 
coding of the text stream could be performed in small 
segments, concurrently with the other steps shown in 
Fig. 10. This option would be feasible only where the 

25 video signal is stored in a format which permits rapid 
access to the closed captioning data without the need 
to play back the entire video presentation in real time 
(e.g. where it is stored in digital format). 

Once the text strings have been built from the 

30 closed captioning data stream, control program 220 
executing on CPU 202 attempts to match each text 
string with the keyword search parameters specified 
by an appropriate profile control block 401 at step 
1002. This matching step is essentially the same as 

35 that performed when in automatic scan mode, and 
shown in Fig. 9. If a match is detected 1003), control 
program 220 causes storage control 210 to retrieve 
a segment of video at which the match was detected 
from storage 105, and display the video segment on 

40 monitor 106 (step 1004). 

At this point, the video presentation system has 
identified a video segment of interest to the viewer, 
and displayed it on monitor 106. The viewer will then 
be offered a plurality of editing options at step 1005. 

45 The viewer may interactively select one or more edit- 
ing options using the keyboard or mouse. These po- 
tential choices are depicted in block 1006, it being un- 
derstood that the list of enumerated choices is not 
necessarily exhaustive. For example, if the television 

so program is being scanned for the purpose of removing 
offensive language and/or scenes, the person editing 
the program may wish to delete audio, to dub audio, 
or to delete audio and video portions of the signal. If 
the editor is creating a new program using a variety 

55 of scenes from old programs, he may copy the video 
and/or audio portions of the signal to some space in 
storage allocated for the new program. The editing 
functions of deleting, copying, inserting, and dubbing, 

11 
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applied to either video or audio portions of the signal, 
are preferably offered to the user. Such editing func- 
tions are known in the art 

When the person performing the editing has 
completed any editing of a video segment that has 5 
been located by control program 220, the control pro- 
gram determines whether additional text strings re- 
main to be analyzed at step 1 007, and if so loops back 
to step 1002 to scan the next text string. When all 
strings have been analyzed, the search/edit function 10 
is complete. 

In the preferred embodiment, an interval is iden- 
tified and saved when there is only a single match 
with the search parameters. However, it would alter- 
natively be possible to require a minimum number of 15 
matches with the search parameters over an interval. 
This may be particularly useful for identifying longer 
video intervals, e.g., identifying entire television pro- 
grams or movies. As will be appreciated by those skil- 
led in the art of computer programming, it would be 20 
relatively simple to maintain a count of the number of 
matches found and identify an interval of television 
presentation as one of interest only if the count ex- 
ceeds some specified number. It will be further un- 
derstood by those skilled in the art of computer data- 25 
base searching that the specification of search para- 
meters could be varied from the simple "AND" and 
m OR" logical relationships of the preferred embodi- 
ment. For example, additional logical operators such 
as TSIOT" could be added; variations of root words 30 
could be searched, etc. 

A preferred embodiment has been described in 
which the hardware is capable of receiving television 
signals from and scanning a single channel at a time. 
However, in an alternative embodiment, it is possible 35 
to receive and scan multiple channels simultaneous- 
ly. In this multiple channel alternative embodiment, it 
will be possible to save video from multiple channels 
simultaneously, and thus the full advantages of scan- 
ning numerous broadcasts for information of interest 40 
in accordance with the present invention can be real- 
ized, at some cost in additional hardware. 

To support the multiple channel scanning em- 
bodiment all hardware must be capable of servicing 
the increased requirements. The bus 201, CPU 202, 45 
storage controller 210 and storage 105 should be ca- 
pable of multiplexing between processing data from 
different channels simultaneously. However, sepa- 
rate tuners 101 and video capture units 207 may be 
required. Additionally, separate closed captioning de- so 
coders 103 may be required. The need for separate 
components to handle each channel being scanned 
would depend on the speed of the device selected, its 
ability to buffer data, etc. It is anticipated that as the 
speed of electronic devices improves and costs de- 55 
dine, the multiple channel alternative embodiment 
will become more attractive. It is further anticipated 
that as a result of closely integrating the various com- 



ponents, it may be necessary to duplicate only small 
portions of the hardware (such as the tuner) to ach- 
ieve multiple simultaneous channel scanning. How- 
ever, it is possible to construct the multiple channel 
embodiment using hardware currently available by 
simply duplicating sufficient hardware. Since bus 201 
available on standard personal computers may be in- 
sufficient to handle the volume of data involved in 
multiple channels, this may require that computer 1 04 
itself be duplicated, in addition to tuner 101, splitter 
1 02, decoder 1 03, etc. 

In the multiple channel scanning embodiment 
control program 220 executes the same steps shown 
in Figs. 8A and 8B with respect to a single channel 
scanning embodiment However, multiple video sig- 
nals from multiple channels are simultaneously con- 
verted to digital and written to buffer 222. Thus, block 
804 of Fig. 8A may be thought of as being repeated 
multiple times in parallel, one for each channel being 
received and converted to video. The appropriate 
steps shown in Fig. 8B are performed separately by 
control program 220 for each channel being scanned. 
Because a match of the keyword search parameters 
is performed separately for each channel, there must 
be a separate match flag and match timer for each 
channel being scanned. As explained previously, con- 
trol program 220 can normally complete steps 81 0- 
838 in much less time than it takes video capture unit 
207 to receive and digitize a segment of video infor- 
mation. Therefore, there is ample time for control pro- 
gram 220 to serially perform the required steps sep- 
arately for each channel being scanned. 

A separate buffer 222 is required for each chan- 
nel being scanned in the multiple channel alternative 
embodiment This need may require a larger memory 
220 than would be required in the single channel pre- 
ferred embodiment A separate segment pointer 325 
is required for each buffer, which means that at step 
814 control program 220 must increment each seg- 
ment pointer, and send a separate command to each 
video capture unit at step 815. 

The profile data structures required to support 
multiple channel scanning have already been descri- 
bed and shown in Figs. 4A-4E. Thus, a single channel 
block 420 exists for each channel to be scanned. 
Since channel blocks contain next channel pointer 
field 422. a chain of channel blocks of indefinite 
length can be specified, enabling the specification of 
mu-ltiple channels. 

A preferred embodiment of this invention has 
been described above, with particular attention to 
hardware components and subassemblies currently 
available in the commercial market It is thus possible 
for one skilled in the art to construct the system de- 
scribed herein from such hardware components. 
However, in an alternative preferred embodiment it 
would be possible to construct the system described 
herein as a single integrated unit In this alternative 
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embodiment, the components shown in Fig. 1 which 
make up system 100 are not necessarily physically 
separated units, but some may exist on a common cir- 
cuit card. Additionally, in this alternative embodiment, 
computer system 104 would not necessarily be a gen- 
eral purpose computer system, but may be a special 
purpose computer dedicated to the functions of the 
presentation system described herein. For example, 
such a system may have only one program it is capa- 
ble of executing, which may be stored in a permanent 
read- only memory (ROM). It is expected that if such 
a presentation system were to be mass-marketed and 
used by viewers unfamiliar with computer operating 
systems, such an integrated unit having a dedicated 
program would be preferable to the system having 
separate components, including a general purpose 
computer, as described in the first preferred embodi- 
ment. 

In the preferred embodiment, the textual stream 
carried in the closed captioning data portion of the 
vertical blanking interval is used as a source of infor- 
mation to be matched with the search parameters. 
The closed captioning data is used in the preferred 
embodiment because it has been adopted as a stan- 
dard within the United States. However, in an alterna- 
tive embodiment, it would be possible to transmit 
other data in the vertical blanking interval or other 
portion of the video signal for decoding and extracting 
information. 

With regard to alternative sources of the text 
stream, it would in particular be possible to employ a 
speech recognition device to extract speech from the 
audio component of the signal and convert this to a 
digital text stream, as shown in Fig. 1 as optional item 
130. While speech recognition unit 130 is shown in 
Fig. 1 as an independent, stand alone unit, it should 
be understood that speech recognition capability 
could be provided by a card which plugs into an avail- 
able slot in computer 104, or alternatively speech rec- 
ognition could be accomplished via suitable software 
stored in memory 203 of computer 104 and executing 
on CPU 202. Many speech recognition devices are 
commercially available at the present time, although 
such devices may have limited vocabularies or accu- 
racy. Even so, such speech recognition devices could 
be used alone or in conjunction with closed captioning 
decoding to provide a text stream. 

As an example of such a use of a speech recog- 
nition device, it is common for closed captioned text 
to contain Simplified "or Vatered-down* versions of 
the actual spoken text of the audio signal. In most cas- 
es, the information content is the same, so decoding 
of the closed captioning signal provides an adequate 
representation of the speech content. But for some 
purposes this is not true. Particularly, closed caption- 
ing text may water-down offensive language. Thus, 
where the present invention is being used as a cen- 
soring device to monitor offensive language, the con- 



current use of a speech recognition device to gener- 
ate additional text to be searched by control program 
220 provides additional accuracy. Because the num- 
ber of offensive words may be small, the limited vo- 

5 cabulary of commercially available speech recogni- 
tion units is not necessarily a handicap. Of course, it 
is anticipated that as speech recognition devices ach- 
ieve greater accuracy and speed, their use in the 
present invention may become more desirable. 

10 The video presentation system of the preferred 

embodiment is designed for use by a television viewer 
and for that reason includes functions such a viewer 
would typically desire. However, in any of various al- 
ternative embodiments, it would be possible to de- 

15 sign special purpose devices having only some of the 
functional capabilities of the video presentation sys- 
tem of the preferred embodiment and/or being de- 
signed for different markets. For example, in an alter- 
native embodiment a video editing facility could be 

20 designed having the search/edit function described 
herein, but without the automatic scan function. Such 
a video editing facility may, for example, be usef ul to 
a television producer to enable editing of television 
programs to be broadcast at a later time. In such an 

25 environment, the automatic scan function would 
probably be considered unnecessary. In other alter- 
natives, the automatic scan function could be present 
without the full range of editing capabilities of the 
search/edit function. 

30 In a further alternative embodiment the video 

presentation system of the present invention could be 
used in conjunction with a video library for accessing 
information from the library. In this alternative, em- 
bodiment, a library would contain a large bank of vid- 

35 eo information for distribution. Distribution could be 
by physical means such as video cassettes, or by on- 
line means from mass storage such as CD-ROM. The 
present invention would be used as a library index for 
finding video material of interest in the video library. 

40 A typical library patron would define a profile indicat- 
ing video of interest. This profile would by used in a 
similar manner to that of the profile described above 
and depicted in Figs. 4A-4E, but may contain other 
fields helpful in searching from a video library. For ex- 

45 ample, the video library may have a system of clas- 
sification, which could be used to narrow the scope 
of the search. The library's computer system, using 
the profile defined by the library patron, would scan 
a closed captioning text stream for matches with the 

so search parameters in the profile. This would require 
that, at the very least, the closed captioning signal be 
available on-line for scanning. When the system lo- 
cates a segment of interest, the segment would pre- 
ferably be displayed to the library patron on a video 

55 monitor, and the patron could then decide whether 
the video selection from which the segment is taken 
is of interest. 

Although a specific embodiment of the invention 
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has been disclosed along with certain alternatives, it 
will be recognized by those skilled in the art that ad- 
ditional variations in form and detail may be made 
within the scope of the following claims. 



Claims 

1 . A television presentation system, comprising: 

a television signal receiver for receiving a 10 
television signal from an external source; 

a closed captioning decoder for extracting 
from said television signal a closed captioning 
stream of textual data; 

means for receiving from a user one or 15 
more search parameters; 

means for searching said stream of textual 
data for occurrences of textual data matching 
said one or more search parameters; 

means for identifying one or more seg- 20 
ments of said television signal, each said seg- 
ment corresponding to an occurrence of said tex- 
tual data matching said one or more search para- 
meters. 

25 

2. The television presentation system of claim 1, 
further comprising: 

means for storing said television signal; 

and 

means for editing said stored television 30 

signal. 

3. The television presentation system of claim 2, 
wherein said means for editing said television sig- 
nal comprises means for deleting portions of said 35 
signal other than said one or more segments cor- 
responding to occurrences of said textual data 
matching said one or more search parameters. 

4. The television presentation system of claim 1 , 40 
further comprising means, responsive to said 
means for identifying one or more segments, for 
storing said one or more segments of said televi- 
sion signal corresponding to occurrences of said 
textual data matching said one or more search 45 
parameters. 

5. An apparatus for facilitating the use of a video 
presentation, comprising: 

means for specifying data of interest to a so 
user of said video presentation; 

means for extracting a text stream from 
said video presentation; 

means for searching said text stream for 
occurrences of said data of interest to a user of 55 
said video presentation; and 

means for extracting one or more seg- 
ments of said video presentation, each said seg- 

14 



ment corresponding to an occurrence of said data 
of interest 

6. The apparatus for facilitating the use of a video 
presentation of claim 5, wherein said means for 
extracting a text stream comprises means for ex- 
tracting textual data from information carried in a 
vertical blanking interval of said video presenta- 
tion. 

7. The apparatus for facilitating the use of a video 
presentation of claim 6, wherein said means for 
extracting a text stream comprises: 

means for extracting a closed captioning 
signal from said video presentation; and 

means for converting said closed caption- 
ing signal to said text stream. 

8. The apparatus for facilitating the use of a video 
presentation of claim 5, wherein said means for 
extracting a text stream comprises a speech rec- 
ognition means for converting speech carried in 
an audio portion of said video presentation into a 
text stream. 

9. A method for identifying television data of inter- 
est, comprising the steps of: 

specifying one or more search parameters 
for identifying television data of interest; 

receiving a television signal; 

decoding a closed captioning portion of 
said television signal to produce a closed caption- 
ing stream of textual data; 

searching said closed captioning stream 
of textual data for data which matches said one 
or more search parameters; 

identifying one or more segments of said 
television signal, each said segment correspond- 
ing to an occurrence of said closed captioning 
stream of textual data matching said one or more 
search parameters. 

10. The method for identifying television data of in- 
terest of claim 9, wherein said one or more search 
parameters comprise one or more key words. 

11. An apparatus for editing a video presentation, 
said video presentation comprising a plurality of 
images intended for sequential display, said ap- 
paratus comprising: 

a video storage for storing said video pre- 
sentation; 

a text extraction device for extracting a 
stream of textual data from said video presenta- 
tion, said textual data representing information 
being conveyed by said plurality of images; 

means for specifying one or more search 
parameters, said search parameters represent- 



27 EP 0 648 054 A2 28 



ing information contained in some portion of said 
video presentation; 

means for searching said stream of textual 
data for occurrences of textual data matching 
said one or more search parameters; s 

means, responsive to said searching 
means, for identifying one or more images repre- 
sented by textual data matching said one or more 
search parameters; 

means for displaying said images identi- 10 
f ied by said identifying means to an operator of 
the editing apparatus; and 

means for interactively editing said dis- 
playing images. 

15 

12. The video editing apparatus of claim 11, wherein 
said text extraction device comprises a closed 
captioning decoder device. 

13. The video editing apparatus of claim 11, wherein 20 
said text extraction device comprises a speech 
recognition means. 

14. A method for locating video data, comprising the 
steps of: 25 

inputting to an automatic locating device 
one or more search parameters representing 
data of interest; 

extracting a text stream from a video pre- 
sentation; 30 

automatically searching, with said auto- 
matic locating device, said text stream for occur- 
rences of text matching said one or more search 
parameters; and 

identifying one or more segments of said 35 
video presentation, each said segment corre- 
sponding to an occurrence of text matching said 
one or more search parameters. 

15. The method of claim 14, wherein said step of ex- 40 
tracting a text stream comprises extracting tex- 
tual data from information carried in a vertical 
blanking interval of said video presentation. 

16. The method for identifying television data of in- 45 
terest of claim 14, wherein said one or more 
search parameters comprise one or more key 
words. 

17. The method for identifying television data of in- so 
terest of claim 14, wherein said step of extracting 

a text stream comprises converting human 
speech carried in an audio portion of said video 
presentation to a text stream. 

55 



15 



BNSOOCID: <EP OB48064A2 I > 



EP 0 648 054 A2 



CO 
O 

1 



O 



en 
o 




v 



I 



CD 
CO 



V 



cc: 

CO 

22 



to 



V 



CD 

co 



UJ 



o 



CD 
O 



O 



UJ 



CO 



in 



o 




ZCD 

2fc 



eg 



CD 



UQOLU 

— »co*— o 

0->CDQ.CJ 
PQUJ— l<CLU 



OuLULU 



M31indS 



a 

UJ 

h-CD 
OO 
LU — I 



co« 



CSJ 



CD Z 



CSI 
CO 

a- 



CO UJ 
CD « LUh- - 

" ""D UJ 
!0 - 

• co> 



HQUJ 

ouu 

CJ 0»— O 
> wCOCO 



16 



EP 0 648 054 A2 




17 



EP 0 648 054 A2 



f— irsJKNcrmior^oocDOf— < M 
OOOOOOOOOi— i»— <i-h 



CSl 



CVJ 
CM 
CM 



PQ 



CNJ 
CNJ 



\ 



\ 



















































































LU 
























Of 
























of 


r— I 


o 
















CM 


r-H 




r— 4 


1— 1 


an 


CO 






in 










C_> 
















i 


1 


i 


1 




1 


1 


i 






i 


i 


























h- 


1— 


*— 


H— 


I— 


h- 


1— 


H— 


»— 


1 — 


i — 


f— 














z 












LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 








S 


















CD 


CD 


CD 




CD 


CD 




CD 


CD 


CD 


CD 


CD 


LU 


LU 


LU 


LU 


LU 


QJ 


LU 


LU 


LU 


LU 


LU 


LU 


CO 


oo 


CO 


CO 


CO 


CO 


CO 


CO 


00 


00 


CO 


CO 


Of 


Of 


of 


Of 


en 


Of 


Of 


Of 


Of 


Of 


Of 


Of 


O 


o 


o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


«c 














« 


<c 












? 














I— 


1— 














<c 




<C 




«X 




<c 












a 


a 












LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 


LU 




> 


> 


> 


> 


> 


> 


> 


> 


> 


> 


% 


§j 


























00 


CO 


00 


CO 


CO 


CO 


00 


CO 


CO 


CO 


00 


CO 


Of 


Of 


of 


Of 


of 


of 


Of 


Of 


Of 


Of 


Of 


Of 


h- 


1— 






h- 


£ 




h— 


h- 




£ 


£ 


a. 


a. 


Q- 




a. 




a. 


Q. 











|LiLiLtLj[jljl^LfLJ Lf "17 



ro 

CD 
Ll. 



in 



I 



oo 



18 



EP 0 648 054 A2 



Field Name 



PROFILE CONTROL BLOCK FORMAT 
Sample Data 



^01 



Title 


'Twins' 


Description 


'MN Twins Baseball News' j 


Interval 


15 


Next Profile PTR 


X '0001D780' 


Last Profile PTR 


X '0001D680' 


Time PTR 


X '0001E338' 


Keyword PTR 


X '0001EC50' 



402 



.403 
404 
405 
406 
407 
.408 



FIG. 4A 



Field Name 



TIME BLOCK FORMAT 
Sample Data 



410 



Date or day 


M T W H F 


Start time 


17:00 


End time 


18:30 


Next Time PTR 


X '0001E350' 


Channel PTR 


X '0001E074' 



411 
412 
413 
414 
415 



FIG. 4B 
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Field Name 



CHANNEL BLOCK FORMAT 
Sample Data 



420 



Channel 


10 


Next Channel PTR 


X '0001E090' 




FIG. 4C 






, OR-ED 
Field Name 


KEYWORD BLOCK 
Sample Data 


FORMAT 


/ 


Next OR-ed PTR 


X '0001F060' 


Key word 


'Minnesota' 


Next AND-ed PTR 


X '0001EC90' 




FIG. 4D 






AND-ED KEYWORD BLOCK 
Field Name Sample Data 


FORMAT 




No. words within 


2 


Key word 


'Twins' 


Next AND-ed PTR 


X '00000000' 



421 
422 



1/431 
432 

433 



440 



,441 
,442 
.443 



FIG. 4E 
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EDIT PROFILE FUNCTION 
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702 
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703 
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Copy Old Profile 
Data Structures 
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Enter New Name 
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Display Profile Values 
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Change Elem't 
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Delete Elem't 
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End Edit 
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SEARCH/EDIT FUNCTION 
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(57) A television presentation and editing system 
uses closed captioning text to locate items of 
interest. A closed captioning decoder (103) ext- 
racts a closed captioning digital text stream 
from a television signal. A viewer specifies one 
or more keywords to be used as search par- 
ameters. A digital processor (104) executing a 
control program scans the closed captioning 
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ing the search parameters. The corresponding 
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operation, the television presentation system 
may be used to scan one or more television 
channels unattended, and save items which 
may be of interest to the viewer. In another 
mode of operation, the system may be used to 
assist editing previously stored video by quickly 
locating segments of interest. 
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1- Claims 1-4,9,10: 

Method for identifying television data of interest, 
xn which a closed caption portion of a received 
television signal is decoded to produce a closed 
caption stream of textual data. This stream is 
searched in order to find data which matches one 
or more search parameters, specified by the user, and 
the segments of the television signal, corresponding 
to these matches are identified in order to be 
processed later on. 

2. Claims 5-8,11-17: 

Method for locating television data, in which a 
stream of textual data is extracted from a 
television signal. This stream is searched in order 
to find data which matches one or more search 
parameters, specified by the user and representing 
data of interest, and the segments of the television 
signal, corresponding to these matches are identified 
in order to be processed later on. 
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